Ensemble Machine Learning Methods Research Articles

A novel ensemble machine learning method for accurate air quality prediction

A novel ensemble machine learning method for accurate air quality prediction

Pre-hospital glycemia as a biomarker for in-hospital all-cause mortality in diabetic patients - a pilot study

BackgroundType 2 Diabetes Mellitus (T2DM) presents a significant healthcare challenge, with considerable economic ramifications. While blood glucose management and long-term metabolic target setting for home care and outpatient treatment follow established procedures, the approach for short-term targets during hospitalization varies due to a lack of clinical consensus. Our study aims to elucidate the impact of pre-hospitalization and intra-hospitalization glycemic indexes on in-hospital survival rates in individuals with T2DM, addressing this notable gap in the current literature.MethodsIn this pilot study involving 120 hospitalized diabetic patients, we used advanced machine learning and classical statistical methods to identify variables for predicting hospitalization outcomes. We first developed a 30-day mortality risk classifier leveraging AdaBoost-FAS, a state-of-the-art ensemble machine learning method for tabular data. We then analyzed the feature relevance to identify the key predictive variables among the glycemic and routine clinical variables the model bases its predictions on. Next, we conducted detailed statistical analyses to shed light on the relationship between such variables and mortality risk. Finally, based on such analyses, we introduced a novel index, the ratio of intra-hospital glycemic variability to pre-hospitalization glycemic mean, to better characterize and stratify the diabetic population.ResultsOur findings underscore the importance of personalized approaches to glycemic management during hospitalization. The introduced index, alongside advanced predictive modeling, provides valuable insights for optimizing patient care. In particular, together with in-hospital glycemic variability, it is able to discriminate between patients with higher and lower mortality rates, highlighting the importance of tightly controlling not only pre-hospital but also in-hospital glycemic levels.ConclusionsDespite the pilot nature and modest sample size, this study marks the beginning of exploration into personalized glycemic control for hospitalized patients with T2DM. Pre-hospital blood glucose levels and related variables derived from it can serve as biomarkers for all-cause mortality during hospitalization.

Benchmarking clinical risk prediction algorithms with ensemble machine learning for the non-invasive diagnosis of liver fibrosis in NAFLD.

Ensemble machine learning methods, like the superlearner, combine multiple models into a single one to enhance predictive accuracy. Here we explore the potential of the superlearner as a benchmarking tool for clinical risk prediction, illustrating the approach in identifying significant liver fibrosis among patients with non-alcoholic fatty liver disease (NAFLD). We used 23 demographic/clinical variables to train superlearner(s) on data from the NASH-CRN observational study (n=648) and validated models with data from the FLINT trial (n=270) and NHANES participants with NAFLD (n=1244). Comparing the superlearner's performance to existing models (FIB-4, NFS, Forns, APRI, BARD, and SAFE), it exhibited strong discriminative ability in the FLINT and NHANES validation sets, with AUCs of 0.79 (95% CI: 0.73-0.84) and 0.74 (95% CI: 0.68-0.79) respectively. Notably, the SAFE score performed similarly to the superlearner, both of which outperformed FIB-4, APRI, Forns, and BARD scores in the validation datasets. Surprisingly, the superlearner derived from 12 base models matched the performance of one with 90 base models. Overall, the superlearner, being the "best-in-class" ML predictor, excelled in detecting fibrotic NASH, and this approach can be used to benchmark the performance of conventional clinical risk prediction models.

Landslide susceptibility mapping using ensemble machine learning methods: a case study in Lombardy, Northern Italy

Landslide susceptibility mapping using ensemble machine learning methods: a case study in Lombardy, Northern Italy

COVID-19 vaccination performance of the U.S. states: a hybrid model of DEA and ensemble machine learning methods

COVID-19 vaccination performance of the U.S. states: a hybrid model of DEA and ensemble machine learning methods

An interpretable framework for modeling global solar radiation using tree-based ensemble machine learning and Shapley additive explanations methods

Machine learning techniques provide an effective and cost-efficient solution for estimating solar radiation for solar energy utilization. However, the reported machine learning-based solar radiation models fail to offer comprehensive explanations for their outputs. Therefore, this study aims to tackle this issue by developing machine learning models that are both accurate and interpretable. To achieve the objective, this study evaluated the performance of tree-based ensemble algorithms, using optimized combinations of model input parameters for different climate zones in China. The results showed that the extreme gradient boosting (XGBoost) models demonstrated the highest overall accuracy, model stability, and generalization ability. At the national scale, the developed XGBoost models yielded an average R2, MAE, and RMSE of 0.939, 1.226 MJ/m2, and 1.663 MJ/m2, respectively, showing significant improvements of 2.13–27.78% in RMSE compared to recently reported models. Most importantly, the state-of-the-art SHapley Additive exPlanations (SHAP) technique was integrated with the developed XGBoost models to enhance model interpretability in terms of global and local feature importance, as well as the interaction effects between model features. The results of the SHAP value analysis demonstrated the robustness of sunshine duration in modeling global solar radiation, revealing thresholds where its values undergo a shift from negative to positive effects on model output. SHAP interaction values illustrated the interaction effects among features in the developed solar radiation model, uncovering the model's complex non-linear relationships. Additionally, this study provided explanations for individual instances based on the SHAP method. Overall, this study provided an accurate, reliable, and transparent machine learning model and an enlightening framework for modeling global solar radiation at sites without observations.

Machine Learning Approaches Predicting the Best Features of Child Birth Using Support Vector Machine (SVM) Algorithm

Cesarean childbirth, though often necessary for medical reasons, carries inherent risks and impacts maternal and neonatal health. Predicting the likelihood of cesarean delivery can assist healthcare providers in making informed decisions and optimizing maternal care. In this study, we propose an ensemble machine learning approach to predict cesarean childbirth based on maternal and fetal health data. First, we collect comprehensive data including maternal age, BMI, gestational age, fetal weight, previous cesarean history, and other relevant factors. After preprocessing the data to handle missing values and encode categorical variables, we select informative features using correlation analysis and domain expertise. Next, we employ various ensemble machine learning methods, including Voting Classifiers, Bagging, Random Forest, Boosting, and Stacking. These methods combine predictions from multiple base learners, leveraging their diverse strengths to enhance predictive accuracy.

Performance Evaluation of Deep, Shallow and Ensemble Machine Learning Methods for the Automated Classification of Alzheimer's Disease.

Artificial intelligence (AI)-based approaches are crucial in computer-aided diagnosis (CAD) for various medical applications. Their ability to quickly and accurately learn from complex data is remarkable. Deep learning (DL) models have shown promising results in accurately classifying Alzheimer's disease (AD) and its related cognitive states, Early Mild Cognitive Impairment (EMCI) and Late Mild Cognitive Impairment (LMCI), along with the healthy conditions known as Cognitively Normal (CN). This offers valuable insights into disease progression and diagnosis. However, certain traditional machine learning (ML) classifiers perform equally well or even better than DL models, requiring less training data. This is particularly valuable in CAD in situations with limited labeled datasets. In this paper, we propose an ensemble classifier based on ML models for magnetic resonance imaging (MRI) data, which achieved an impressive accuracy of 96.52%. This represents a 3-5% improvement over the best individual classifier. We evaluated popular ML classifiers for AD classification under both data-scarce and data-rich conditions using the Alzheimer's Disease Neuroimaging Initiative and Open Access Series of Imaging Studies datasets. By comparing the results to state-of-the-art CNN-centric DL algorithms, we gain insights into the strengths and weaknesses of each approach. This work will help users to select the most suitable algorithm for AD classification based on data availability.

Predicting Elimination of Small-Molecule Drug Half-Life in Pharmacokinetics Using Ensemble and Consensus Machine Learning Methods.

Half-life is a significant pharmacokinetic parameter included in the excretion phase of absorption, distribution, metabolism, and excretion. It is one of the key factors for the successful marketing of drug candidates. Therefore, predicting half-life is of great significance in drug design. In this study, we employed eXtreme Gradient Boosting (XGboost), randomForest (RF), gradient boosting machine (GBM), and supporting vector machine (SVM) to build quantitative structure-activity relationship (QSAR) models on 3512 compounds and evaluated model performance by using root-mean-square error (RMSE), R2, and mean absolute error (MAE) metrics and interpreted features by SHapley Additive exPlanation (SHAP). Furthermore, we developed consensus models through integrating four individual models and validated their performance using a Y-randomization test and applicability domain analysis. Finally, matched molecular pair analysis was used to extract the transformation rules. Our results revealed that XGboost outperformed other individual models (RMSE = 0.176, R2 = 0.845, MAE = 0.141). The consensus model integrating all four models continued to enhance prediction performance (RMSE = 0.172, R2 = 0.856, MAE = 0.138). We evaluated the reliability, robustness, and generalization ability via Y-randomization test and applicability domain analysis. Meanwhile, we utilized SHAP to interpret features and employed matched molecular pair analysis to extract chemical transformation rules that provide suggestions for optimizing drug structure. In conclusion, we believe that the consensus model developed in this study serve as a reliable tool to evaluate half-life in drug discovery, and the chemical transformation rules concluded in this study could provide valuable suggestions in drug discovery.

A serial image analysis architecture with positron emission tomography using machine learning combined for the detection of lung cancer

Introduction and objectivesLung cancer is the second type of cancer with the second highest incidence rate and the first with the highest mortality rate in the world. Machine learning through the analysis of imaging tests such as positron emission tomography/computed tomography (PET/CT) has become a fundamental tool for the early and accurate detection of cancer. The objective of this study was to propose an image analysis architecture (PET/CT) ordered in phases through the application of ensemble or combined machine learning methods for the early detection of lung cancer by analyzing PET/CT images. Material and methodsA retrospective observational study was conducted utilizing a public dataset entitled “A large-scale CT and PET/CT dataset for lung cancer diagnosis.” Various imaging modalities, including CT, PET, and fused PET/CT images, were employed. The architecture or framework of this study comprised the following phases: 1. Image loading or collection, 2. Image selection, 3. Image transformation, and 4. Balancing the frequency distribution of image classes. Predictive models for lung cancer detection using PET/CT images included: a) the Stacking model, which used Random Forest and Support Vector Machine (SVM) as base models and complemented them with a logistic regression model, and b) the Boosting model, which employed the Adaptive Boosting (AdaBoost) model for comparison with the Stacking model. Quality metrics used for evaluation included accuracy, precision, recall, and F1-score. ResultsThis study showed a general performance of 94% with the Stacking method and a general performance of 77% with the Boosting method. ConclusionsThe Stacking method proved to be a model with high performance and quality for lung cancer detection when analyzing PET/CT images.

Automated Classification of COVID-19 Chest X-ray Images Using Ensemble Machine Learning Methods

This study delves into the efficacy of ensemble machine learning techniques for classifying chest X-ray images into three distinct categories: Normal, COVID-19, and Lung Opacity. Employing the Random Forest Classifier and a rigorous k-5 cross-validation framework, we aimed to enhance diagnostic accuracy for one of the most urgent medical challenges today—rapid and reliable COVID-19 detection. The analysis revealed an average accuracy of 51%, with varying precision and recall across different folds. The F1-score remained consistently around 35%, indicating a need for improved balance between precision and recall. Visualizations such as performance metric trends and a confusion matrix provided further insight into the classifier's performance, highlighting a notable degree of misclassification. Despite moderate success in the automated classification of the images, our research illustrates the complexity of applying machine learning to medical imaging, especially in differentiating between diseases with overlapping radiographic features. The study’s findings emphasize the potential of machine learning models to support diagnostic processes and suggest the necessity of advanced pre-processing techniques and extended datasets for enhanced model training. The research contributes to the growing body of knowledge in computational diagnostics and underscores the importance of developing robust, accurate machine learning tools to aid in the global healthcare crisis precipitated by the pandemic.

Comparison of SARIMA, Bagging Exponential Smoothing with STL Decomposition and Robust STL Decomposition for Forecasting Red Chili Production

Time series analysis enables the identification of trends and patterns in data, allowing for the development of forecasting models that predict future values. One effective approach for forecasting seasonal time series data is the Seasonal Autoregressive Integrated Moving Average (SARIMA) method. Bagging Exponential Smoothing with STL Decomposition (BES-STL) is an ensemble machine learning method aimed at enhancing forecasting accuracy. STL Method, which stands for Seasonal-Trend decomposition using Loess, is utilized to decompose time series data into three components, namely trend, seasonal, and remainder components. In the remainder component, the process of bootstrap aggregation (bagging) with Moving Block Bootstrapping (MBB) is used to obtain synthetic data, followed by averaging the value by month from the entire series as the forecast results. A comparative analysis was conducted using the SARIMA, BES-STL, and BES-RSTL models. The optimal model, with the lowest MAPE and RMSE, is then implemented to predict national red chili production. The results indicate that the SARIMA(1,1,1)(0,1,1)12 model has the best performance with a MAPE of 7.06 and a RMSE of 95,473. The top-performing model is utilized to forecast data from January to December 2022. Additionally, the forecasted results are compared to the actual data, resulting in a highly precise MAPE of 5.39.

The Data Product-service Composition Frontier: A Hybrid Learning Approach

The service dominant logic is a base concept behind modern economies and software products, with service composition being a well-known practice for companies to gain a competitive edge over others by joining differentiated services together, typically assembled according to a number of features. At the other end of the spectrum, product compositions are a marketing device to sell products together in bundles that often augment the value for the customer, e.g., with suggested product interactions, sharing, and so on. Unfortunately, currently each of these two streams—namely, product and service composition—are carried out and delivered individually in splendid isolation: anything is being offered as a product and as a service, disjointly. We argue that the next wave of services computing features more and more service fusion with physical counterparts as well as data around them. Therefore a need emerges to investigate the interactive engagement of both (data) products and services. This manuscript offers a real-life implementation in support of this argument, using (1) genetic algorithms (GA) to shape product-service clusters, (2) end-user feedback to make the GAs interactive with a data-driven fashion, and (3) a hybridized approach which factors into our solution an ensemble machine-learning method considering additional features. All this research was conducted in an industrial environment. With such a cross-fertilized, data-driven, and multi-disciplinary approach, practitioners from both fields may benefit from their mutual state of the art as well as learn new strategies for product, service, and data product-service placement for increased value to the customer as well as the service provider. Results show promise but also highlight plenty of avenues for further research.

Stacked ensemble modeling for improved tuberculosis treatment outcome prediction in pediatric cases

SummaryThe promising results of ML (machine learning) methods in various disciplines have led to the frequent use of these methods in health fields such as disease diagnosis, personalized medicine, medical image‐based diagnosis, and predicting the number of deaths and cases in a pandemic. However, a neglected area in the field of healthcare is the lack of study with ML to predict treatment outcomes for tuberculosis (TB) patients, particularly children experiencing failed treatment. This need has become more apparent as the coronavirus pandemic has reversed the gains of health institutions with TB disease, especially in children. Therefore, this article conducted a study using the stacked ensemble ML method to early predict the risk for children experiencing a failed treatment outcome of TB. To fulfill this need and determine the most appropriate technique, a two‐stage methodology was followed in this work. First, predictions were obtained by combining the information gain feature selection (IGFS) approach with a variety of single‐based ML algorithms, including logistic regression (LR), deep belief neural networks (DBN), random forest (RF), and decision tree (DT). Second, the proposed method, which includes a stacked ensemble ML technique, was used. The latter model uses LR as a meta‐learner and the aforementioned single‐based ML algorithms (DBN, LR, RF, and DT). The performance results of ML models used in the two stages were compared, and the proposed model which is the combination of the stack‐based ensemble learning model and the IGFS technique provided better ROC curves, accuracy, precision, and recall results.

Refined landslide susceptibility mapping in township area using ensemble machine learning method under dataset replenishment strategy

Landslide susceptibility mapping (LSM) warrants considerable attention as a prerequisite for risk assessment and prevention in response to the increasing global occurrences of landslides. However, refined LSM at the township scale still faces the problem of the limited number of landslides and the oversight of relevant factors associated with different landslide types. This work is dedicated to propose a novel dataset replenishment strategy designed to address landslide sample scarcity, and to identify dominant factors in high landslide susceptibility zones to refine LSM at township scale for decision makers. Herein, the Wuling County in the Three Gorges Reservoir area of China was taken as taken as the research case. Initially, field investigation and calibration of 21 landslides yielded accumulation and rock areas, and the corresponding impact factors were therefore proposed to construct input features for machine learning facilitated by factor diagnosis using Pearson correlation coefficient (PCC) and multicollinearity test. Subsequently, a training and validation dataset split of 8: 2 was randomly generated, enabling the application of ensemble machine learning model for LSM prediction. The study culminated in the feasibility of dataset replenishment strategy, which was employed to explore uncertainties and identify governing factors in LSM. The results indicate that the bagging-ANN model exhibits superior performance in both accumulation and rock areas, with Areas Under Curve (AUC) of 0.987 and 0.994. Furthermore, the dataset replenishment strategy emerges as instrumental in enhancing the reliability of LSM, as evidenced by an 8.97 % reduction in average value (AVG) and a 11.00 % increase in standard deviation (STD). The governing factors for landslides in accumulation and rock areas are designated as river erosion and height, respectively. Together with the excellent performance, this study is expected to provide a promising reference for LSM in other townships worldwide.

StresSense: Real-Time detection of stress-displaying behaviors

BackgroundWrist-worn gadgets like smartphones are ideal for unobtrusively gathering user data, in various fields such as health and fitness monitoring, communication, and productivity enhancement. They seamlessly integrate into users' daily lives, providing valuable insights and features without the need for constant attention or disruption. In sensitive domains like mental health, these devices provide user-friendly, privacy-protected means of diagnosis and treatment, offering a secure and cost-effective avenue for seeking help. ObjectivesThis study addresses the limitations of traditional mental health assessment techniques, such as intrusive sensing and subjective self-reporting, by harnessing the unobtrusive data collection capabilities of smartphones. Equipped with accelerometers and other sensors, these devices offer a novel approach to mental health research. Our objective was to develop methods for real-time detection of stress and boredom behavior markers using smart devices and machine learning algorithms. MethodologyBy leveraging data from accelerometers (A), gyroscopes (G), and magnetometers (M), we compiled a dataset indicative of stress-related behaviors and trained various machine-learning models for predictive accuracy. The methodology involved collecting data from motion sensors (A, G, and M) on the dominant arm's wrist-worn smartphone, followed by data preprocessing, transformation from time series format, and training a Deep Neural Network (DNN) model for activity recognition. FindingsRemarkably, the DNN achieved an accuracy of 93.50% on test data, outperforming traditional and ensemble machine learning methods across different window sizes, and demonstrated real-time accuracy of 77.78%, validating its practical application. ConclusionIn conclusion, this research presents a novel dataset for detecting stress and boredom behaviors using smartphones, reducing reliance on costly devices and offering a more objective assessment. It also proposes a DNN-based method for wrist-worn devices to accurately identify complex activities associated with stress and boredom, with benefits in terms of privacy and user convenience. This advancement represents a significant contribution to the field of mental health research, providing a less intrusive and more user-friendly approach to monitoring mental well-being.

ICE-VDOP: an integrated clustering and ensemble machine learning methods for an enhanced vector-borne disease outbreak prediction using climatic variables

ICE-VDOP: an integrated clustering and ensemble machine learning methods for an enhanced vector-borne disease outbreak prediction using climatic variables

Earth fissure susceptibility mapping: Application of random subspace‐based novel ensemble approaches

The development of earth fissures, which are linear fractures with openings or offsets on the land surface, can severely affect landforms, especially in urban areas, in the form of earthquakes causing major concern on human lives as well as damage to infrastructures. Thus, an early warning map for lands susceptible to earth fissures can better equip planners for formulating mitigation strategies. In this study, we focus on the Damghan Plain in Iran for preparation of earth fissure susceptible maps using several topographical, hydrological, geological and environmental conditioning factors. In order to train these conditioning factors and preparation of earth fissure susceptibility maps, 124‐earth fissure field‐based samples, for training and validation purposes, were used by random subspace (RS) model based on four other machine learning ensemble methods such as RS‐Naïve‐Bayes Tree (NBTree), RS‐alternating decision tree (ADTree), RS‐Fisher's Linear Discriminant Function (FLDA) and RS‐Logistic model tree (LMT). From the validation technique, the receiver operating characteristic (ROC) curve performance test demonstrates that the RS‐NBTree model was the best suited with area under curve (AUC) = 0.974 followed by RS‐ADTree (AUC = 0.966), RS‐LMT (AUC = 0.954), RS‐FLDA (AUC = 0.948) and RS (AUC = 0.923). The results from our study can be useful for environmental management and risk reduction.

Comprehensive evaluation of Mal-API-2019 dataset by machine learning in malware detection

This study conducts a thorough examination of malware detection using machine learning techniques, focusing on the evaluation of various classification models using the Mal-API-2019 dataset. The aim is to advance cybersecurity capabilities by identifying and mitigating threats more effectively. Both ensemble and non-ensemble machine learning methods, such as Random Forest, XGBoost, K Nearest Neighbor (KNN), and Neural Networks, are explored. Special emphasis is placed on the importance of data pre-processing techniques, particularly TF-IDF representation and Principal Component Analysis, in improving model performance. Results indicate that ensemble methods, particularly Random Forest and XGBoost, exhibit superior accuracy, precision, and recall compared to others, highlighting their effectiveness in malware detection. The paper also discusses limitations and potential future directions, emphasizing the need for continuous adaptation to address the evolving nature of malware. This research contributes to ongoing discussions in cybersecurity and provides practical insights for developing more robust malware detection systems in the digital era.

Uncertainty Assessment of Surface Water Salinity Using Standalone, Ensemble, and Deep Machine Learning Methods: A Case Study of Lake Urmia

Uncertainty Assessment of Surface Water Salinity Using Standalone, Ensemble, and Deep Machine Learning Methods: A Case Study of Lake Urmia